18.05 Lecture 18 
March 18, 2005 



Law of Large Numbers. 

Xi, ...,Xn - i.i.d. (independent, identically distributed) 



_ Xi + ... + Xn 

x= > asn^(X),lliAi 

n 

Can be used for functions of random variables as well: 
Consider Yi = r{Xi) - i.i.d. 

Y = — — as n ^ oo,EYi = EriXi) 

n 

Relevance for Statistics: Data points x^, as n ^ oo, 

The average converges to the unknown expected value of the distribution which often contains a lot (or 
all) of information about the distribution. 

Example: Conduct a poll for 2 candidates: 

p e [0, 1] is what we're looking for 

Poll: choose n people randomly: Xi, ...,X„ 

F{Xi = l)=p 

F{Xi = 0) = l-p 

EXi = l(p) + 0(1 -p)=p^ Xi + ... + Xn ^^^^ 

n 

Other characteristics of distribution: 

Moments of the distribution: for each integer, k > 1, kth moment EX'^ 

kth moment is defined only if E|X|*' < oo 

Moment generating function: consider a parameter y G M. 

and define (/)(t) = Ee*^-^ where X is a random variable. 

(j){t) - m.g.f. of X 

Taylor series of (j){t) = ^ ^"^^'^ 

fc=0 

Taylor series of Ee*^ = E V = V ^-EX" 

k=0 fe=0 

EX'' = 0'=(O) 

Example: Exponential distribution E{a) with p.d.f. f{x) = {ae~°'^,x > 0;0,x < 0} 

Compute the moments: 

EX'^ = x''ae~'^^dx is a difficult integral. 

Use the m.g.f.: 

/■oo /-oo 

0(i)=Ee*^= / e*^ae-"^da;= / ae(*-"'^da; 
Jo Jo 

(defined if t < oo to keep the integral finite) 
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= 1 _ = = 1 = Vf-)^ = V -EX'' 

i-a '° t-a a-t 1-t/a a' ^ k\ 

' fe=0 



Recall the formula for geometric series: 

oo ^ 

Ex'' = when k < 1 
1 — rr 



k=0 



1 Ex'' ^ . k\ 

= ^-i ^Ex" = —r 

a" fe! a" 



The moment generating function completely describes the distribution. 

Ex'' = J x''f{x)dx 

If f(x) unknown, get a system of equations for f unique distribution for a set of moments. 
M.g.f. uniquely determines the distribution. 

Xi, X2 from E(a), Y = Xi + X2. 

To find distribution of sum, we could use the convolution formula, 
but, it is easier to find the m.g.f. of sum Y: 

Ee*^ = Ee*(^i+^^^ = Ee^^^e*^^ = Ee*^iEe*^= 
Moment generating function of each: 

a 
a — t 

For the sum: 



Consider the exponential distribution: 

E{a) ~ Xi, EX = ^, f{x) = {ae-""", a; > 0; 0, a; < 0} 

This distribution describes the life span of quality products. 
a = if a small, life span is large. 

Median: 

TO € R such that: 

P(X > to) > < m) > ^ 

X >^ 

P= 1/3 1/3 1/3 

(There are times in discrete distributions when the probability cannot ever equal exactly 0.5) 
When you exclude the point itself: P(X > m) <\ 

P(X < to) + P(X > TO,) = 1 

The median is not always uniquely defined. Can be an interval where no point masses occur. 
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X X K X 

1/4 1/4 ™ 1/4 1/4 

For a continuous distribution, you can define P > or < m as equal to |. 
But, there are still cases in which the median is not unique! 




For a continuous distribution: 

F{X < m) = F{X >m) = ^ 



The average measures center of gravity, and is skewed easily by outliers. 




The average will be pulled towards the tail of a p.d.f. relative to the median. 
Mean: find a € R such that E(X — a)^ is minimized over a. 

^E(X - af = -E2{X - a) = 0,EX - a = ^ a = EX 
expectation - squared deviation is minimized. 

Median: find a G M such that E\X — a\ is minimized. 
E\X — a\> E\X — m\, where m - median 

E(|X-a| - |X-to|) > 
/ (|a; — a| — |a; — m\)f{x)dx 
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Need to look at each part: 

1) a — X — {m — x) = a — m,x < m 

2) x — a — {x — m) = m — a,x > m 

3) a — x — {x + m) = a + m — 2x, m < x < a 




The integral can now be simplified: 

(|a; — a| — |a; — m|)/(x)da: > / {a — m)f{x)dx+ / {m — a)f{x)dx = 

J — oc J m 

/m noo 
f{x)dx - / f{x)dx) = (a - m){V{X < m) - ¥{X > m)) > 
-oo Jm 

As both (a — ni) and the difference in probabilities are positive. 
The absolute deviation is minimized by the median. 

** End of Lecture 18 
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